Monitoring is increasingly used in various applications such as business performance analytics, RFID tracking, and analyzing signals from financial indicators and strategies. In many monitoring applications events are emitted, stored, and processed by different components. For example, in business performance monitoring streams of events provide real-time information that is processed, analyzed, and aggregated while crossing different layers of abstractions: from the lower IT layer to the highest business layer. Queries can span more than one such layer, while the processing itself is enabled by multiple components: event bus, various correlation engines, and dedicated monitors.
A continuous monitoring query can be deployed in various configurations of the monitoring system for optimizing the monitoring system. Many optimization methods focus on choosing a query configuration that minimizes total latency and/or work. However, minimizing latency and/or work dos not maximize throughput of the system. Also, each operator of a continuous query requires a certain amount of execution time for every incoming data tuple, which leads to an upper bound on the rate at which tuples can be processed. If the input streams exhibit higher rates than the query operators can process, then special mechanisms need to be in place to handle them.
When high input rates represent only short bursts, buffers can be used to temporarily store the overflow of incoming data. If, instead, the high rates have to be supported for a long period of time, then data needs to be purged out of the input to the operators. This approach cannot avoid the deterioration of the quality of query results. One method for determining which events to shed in order to return a high-quality result is load shedding. However, some loss of quality is unavoidable when information is discarded. For some applications any event may contain critical information and reduction in the quality of results still occurs even with load shedding.
Therefore a need exists to overcome the problems with the prior art as discussed above.